Analysis of Network Traffic

Bartosz Przybylski

If you administer a network of any kind you can be certain that sooner or later it will become a target of an attack. However, you are capable of eliminating, or at least significantly reducing any chances of its success. This is led to by many ways: from disabling services through firewalls to IDSes; still, it may just turn out that the most important issue here is the ability of distinguishing traffic of "good" and "bad" packets.

Pcap is one of the most commonly used libraries for coding network traffic. It offers extremely fine-grained access to individual ISO/OSI layers. Its other advantage is its availability for many different operating systems (more about it in hakin9 5/2005 no. 14, in the article by Konrad Malewski titled Full Control, or Low-level Access to the Network) and programming languages.

A Hammer, a Screwdriver, a sniffer – or, Analysis Tools

It simply won't do to consider network analysis without tools which will make our life much easier. Let us begin with sniffers.

Ethereal

Ethereal is one of the best-known network analysis tools. It offers many features which are of high use during an analysis. The two dominating characteristics of this projects are: storing network traffic in various container format and a graphical user interface. Although the latter is not exactly necessary for analysis, it is a nice addition which makes work easier.

Tcpdump

Tcpdump is another, very good sniffer. It is by the authors who wrote the Pcap library. The program can work with several unofficial front-ends, it has however been designed primarily to be used directly from the command line (the shell) of the system.

Our Armaments

Ahead you will find a list of programs and scripts which we'll be using while analysing network traffic:

• capinfos (a part of the ethereal suite)

• tcpdstat,

• zonk.pl (a simple script for network administrators – written by the author of the article),

• a handful of custom scripts.

Listing 1. Verifying authenticity of two files (aut.sh)

#!/bin/sh
 if [ -z $2 ]; then
 echo "Usage: $0 authentic_file file_for_check";
exit
 if
 md5sum $1 | cut -c1-32 > /tmp/f1.cksum
 md5sum $2 | cut -c1-32 > /tmp/f2.cksum
 sha1sum $1 | cut -c1-40 >> /tmp/f1.cksum
 sha1sum $2 | cut -c1-40 >> /tmp/f2.cksum
 res=`/usr/bin/cmp /tmp/f1.cksum /tmp/f2.cksum`
if [ -z "$res" ]; then
 echo "File is authentic"
 else
 echo "File is not authentic"
 fi
 rm /tmp/f1.cksum /tmp/f2.cksum

Warranty of Authenticity

If the results of a conducted analysis are to serve as evidence against the attacker, it is important for us to prove the authenticity of the file network traffic has been logged in, as well as the one our analysis has been based on.

It is important for the original file, the one which will serve as primary evidence, to have the time stamp as close to the intrusion time as possible. In order to ensure this one ought to copy the traffic log file to a separate directory set read-only permissions on both the file and the directory. We do it the following way:

mkdir ~/analyze

cp ./traffic.cap ~/analyze

chmod 444 ~/analyze/traffic.cap ~/analyze/

Once we have obtained a protected copy our evidence will be based on, we must also prove its authenticity; we will be helped here the simple script from Listing 1.

This script can also become useful to prove authenticity of the intercepted traffic. It will be even more convincing to store similar checksums for the intercepted traffic.

Listing 2. Output of capinfo

$ capinfo traffic.cap
 1    File name: traffic.cap
 2    File type: libpcap (tcpdump, Ethereal, etc.)
 3    Number of packets: 1194
 4    File size: 93506 bytes
 5    Data size: 213308 bytes
 6    Capture duration: 342.141581 seconds
 7    Start time: Thu Jun 23 14:55:18 2005
 8    End time: Thu Jun 23 15:01:01 2005
 9    Data rate: 623.45 bytes/s
 10    Data rate: 4987.60 bits/s
 11    Average packet size: 178.65 bytes

Network Analysis

Let us now proceed to our primary task, that is – analysis of intercepted network traffic. In order to perform such an analysis one must gather basic information about intercepted traffic. For that we shall use capinfo, which is a part of the ethereal suite.

Let's have a closer look at Listing 2 and think about what kind of information we will obtain thanks to capinfo.

We ignore the first line, as it only contains the file name. The second contains information about the format of the file. In this case, the file has been in pcap format. Another, also quite popular system for storing network traffic is Microsoft Network Monitor x.x, where x.x is the version of the library; the most widespread one is version 2.x. These are of course by no means the only formats for storing network traffic in files.

The following line of (3) the output lists the number of packets which "flowed" through our network while sniffing. Next, we have got the size of the file (4) and the amount of data our recorded traffic consists of (5). These are followed consecutively by: the exact duration of sniffing (6), date and time of the beginning (7) and the end (8) of sniffing, average data flow in bytes (9) and bits (10) per second, whereas the final line (11) shows us the average size of a packet.

Listing 3. Output of tcpdstat

DumpFile: traffic.cap
 FileSize: 0.09MB
 Id: 200506231455
 StartTime: Thu Jun 23 14:55:18 2005
 EndTime: Thu Jun 23 15:01:01 2005
 TotalTime: 342.14 seconds
 TotalCapSize: 0.07MB CapLen: 68 bytes
 # of packets: 1194 (208.31KB)
 AvgRate: 5.08Kbps stddev:30.22K
 ### IP flow (unique src/dst pair) Information ###
# of flows: 66 (avg. 18.09 pkts/flow)
 Top 10 big flow size (bytes/total in %):
20.0% 16.3% 15.7% 12.9% 4.8% 4.0% 2.9% 1.3% 1.3% 1.2%
### IP address Information ###
# of IPv4 addresses: 68
 Top 10 bandwidth usage (bytes/total in %):
69.9% 21.5% 18.5% 17.5% 16.9% 13.9% 5.4% 5.2% 4.5% 4.3%
# of IPv6 addresses: 4
 Top 10 bandwidth usage (bytes/total in %):
81.5% 59.2% 40.8% 18.5%
### Packet Size Distribution (including MAC headers) ###
<<<<
[ 32- 63]: 857
 [ 64- 127]: 104
 [ 128- 255]: 79
 [ 256- 511]: 61
 [ 512- 1023]: 14
 [ 1024- 2047]: 79
 ### Protocol Breakdown ###
<<<<
protocol packets bytes bytes/pkt
 ------------------------------------------------
 [0] total 1194 (100.00%) 213308 (100.00%) 178.65
 [1] ip 988 ( 82.75%) 198381 ( 93.00%) 200.79
 [2]   tcp 884 ( 74.04%) 180408 ( 84.58%) 204.08
 [3]    http(s) 219 ( 18.34%) 124825 ( 58.52%) 569.98
 [3]    other 665 ( 55.70%) 55583 ( 26.06%) 83.58
 [2]   udp 94 ( 7.87%) 17247 ( 8.09%) 183.48
 [3]    dns 9 ( 0.75%) 2752 ( 1.29%) 305.78
 [3]    other 85 ( 7.12%) 14495 ( 6.80%) 170.53
 [2] icmp 7 ( 0.59%) 546 ( 0.26%) 78.00
 [2] igmp 3 ( 0.25%) 180 ( 0.08%) 60.00
 [1] ip6 5 ( 0.42%) 422 ( 0.20%) 84.40
 [2] icmp6 5 ( 0.42%) 422 ( 0.20%) 84.40

Already at this moment of the analysis one can draw certain conclusions with respect to possible unauthorised traffic (especially in case of large companies). If one observes a suspiciously large number of packets or high average size of packets, in a short time interval (i.e. for relatively small values of Capture duration), one can suspect that our network is hosting programs assisting downloads from the Internet. Then again, it doesn't always have to be the truth – such traffic could also be caused e.g. by fetching program updates.

The next step is to examine the properties of traffic in more detail, this time at the level of packets; we are interested in the type of the protocol and the size of the packet. We shall also learn much about characteristics of our bandwidth usage; for that purpose we will use a modified version of the program tcpdstat by Dave Dittrich, based on the example shown on Listing 3.

Amounts of packets in different size bins as well as more detailed information pertaining to traffic protocols (this information can be found following the title Protocol Breakdown). Individual lines are shown in the following format:

[protocol "level"] protocol number_of_packets (percentage_of_all_packets) number_of_bytes (percentage_of_total) average_number_of_bytes_per_packets.

With this information one can find out what packets from different protocol families pass through the network. We should make note of that, as based on the analysis of protocols listed here, we can make conclusions as to whether or not our network is the object of a remote analysis or under attack.

Note the large amount of captured TCP packets. Although information about HTTP packets do not cause us much anxiety (unless it's someone maliciously exploiting our PHP applications), we should think about what could have caused such a high traffic in the category other of TCP packets. This traffic has been defined in pcap according to the network layer, but not defined depending on ports. Most commonly it is output traffic, but not always. It could also be an attempt of scanning out system.

Other information presented to us by tcpdstat is e.g. 10 highest load occurrences in the network. However, such information is only of use to us when we generate statistics.

Now that we've got information about packets and protocols, it would pay to learn more about recipients and senders of those packets. For that one can use the simple script from Listing 4.

On its output the script will present us IP addresses (albeit only source ones) of packets passing through our station. That information alone would suffice to prepare firewall rules accepting or rejecting certain IP addresses. One should also find out whether any of the addresses belong to bogon space.

If that is the case, it means our attack is or has been subject to a failed DDoS. Under such circumstances one should use for a few days a script which blocks bogon addresses on the firewall; such a script is available on the Web site http://completewhois.com/).

A minor change in searchip will provide us with information about MAC addresses of interfaces analysed packets come from. Then again, this is not necessary here – we are analysing traffic originating from the Internet, not from the local network.

When it comes to DoS and DDoS attacks, their subject is so extensive that one would have to write a whole separate article about them. In hakin9 issue 5/2004 Andrzej Nowak and Tomasz Pot�ga describe how to protect yourself from such an attack taking place.

One should add here that blocking addresses used in DDoS attacks that have already occurred does not make much sense, as the attacker(s) is/are certain to use a different address pool next time.

HTTP, FTP – Analysis of Information

At this point, let us ask ourselves the following questions: are we able to prevent an in-depth analysis of our Web server, along with further consequences of such an operation? Can we prevent out FTP from being used for port scanning? The answer is "yes", but this is not a simple task. When performing it by hand one will find out that it takes an awful lot of time. Let us conduct a part of such an analysis to learn how time-consuming it is.

Open our file with captured traffic using the ethereal suite (we use it primarily because of its graphical user interface). Select one of the packets initiating transmission of documents over WWW (the body of such a packet must contain the phrase GET or HEAD in its body).

Example of such a packet has been presented in Figure 1. As one can see, every "good" connection contains information passed automatically by the browser. These include e.g.: the type and the version of the browser (User-agent), as well as the accepted type of files (Accept), encoding (Accept-Encoding) and preferred language (Accept-Language). Appearance of information about how long the connection is to be kept alive, as well as identifiers of cookies, can appear here as well.

Listing 4. A script for extracting various IP addresses from a pcap file (searchip.pl)

#!/usr/bin/perl   use Switch;
use Net::Pcap;
my $err;
my $rep;
my $curr_ip;
my @ip_table = ();
sub connread;
if ($ARGV[0] eq "") 
{ print "Usage: ./search_ip ";
exit;
}   if($pcap = Net::Pcap::open_offline($ARGV[0], $err)) # open the file { Net::Pcap::loop($pcap, -1, &connread, ''); # pass each packet to the connread function } else 
{ print "Errorn";
exit;
}   print "Found different IP:n";
foreach $ip_table(@ip_table) 
{ print $ip_table."n";
}   sub connread # the main function { my($data, $header, $packet) = @_;
my $packet = unpack('H*', $packet);
$rep=0;
# locate the IP in the packet if ($packet =~ m/^w{44}(..)(..)(w{4}|w{8})(..)(..)(..)(..)w{8}(....)(....)w+(.*)/) { # store it in a variable and compare it to already-added ones $curr_ip = hex($4).".".hex($5).".".hex($6).".".hex($7);
foreach $ip_table(@ip_table) 
{ if($ip_table eq $curr_ip) { $rep = 1;
} } if ($rep == 0) { # if no such IP yet, add it to the array push(@ip_table, $curr_ip);
} } }

Due to servers eliminating exceedingly idle connections, virtually all network attackers do not provide information included by default by browsers, whilst analysing information made available by the server (its banners). We can take advantage of this fact.

Should incomplete headers be found while scanning the traffic, we can presume that our server has been a subject of an analysis and an attack is soon to follow. Unfortunately (or maybe luckily) this is not always true, as console browsers (e.g. Lynx, links) do not leave "visiting cards". It is therefore possible to make a mistake; then again those browsers do supply cookie information, which sets them apart from attackers examining server banners.

Now, let us consider FTP. If one provides an anonymous FTP server, it can be used to scan ports of any other server thanks to the command PORT. The command can be disabled of course, but this will also make it impossible to establish active connections to our server. If active connections are needed for some reason, one had better periodically perform analyses of FTP traffic on the server. Just like with HTTP we could spend hours on scanning traffic and block individual IP addresses at the firewall, on the other hand we are concerned about performance. To our aid in both cases comes the script fhelp.pl (see Listing 5).

This simple script scans through the given file in pcap format or works "live", picking out lack of browser information in HTTP packets. It also analyses FTP traffic in search for PORT commands. Having found such traffic, it asks the user whether the relevant IP address should be blocked at the firewall (it works only with iptables). It is obvious that the script must be run as the root user.

SSL – the Achilles' Heel

Everyone who performs network analyses sooner or later encounters encrypted traffic. This is a classical problem of most analyses.

While it is possible to decrypt this traffic, experience shows this is entirely not worth the effort. To begin with, in 999 out of 1000 cases, this traffic is legitimate. Second, decryption of those packets would take an unreasonably long time – it would require finding the sequence key for each transmission. Having thought the matter of these kinds of transmissions over thoroughly, one will quickly arrive at the conclusion that the only sensible solution here is to allow encrypted connections from trusted hosts only. Unfortunately this simply won't work if the server in question is to be publicly available and offer encrypted connections to clients of its services by definition.

Lazy Employees

At this point it will be worth it to mention the script zonk.pl (by the author of the article).

This simple application based on conditional expressions and the pcap library, scans current network traffic for simple abuses of bandwidth (p2p, im and irc traffic) based on server port numbers used by the "forbidden transmissions". The script can prove useful for administrators of corporate networks, assisting them in searching and elimination of excessive or unauthorised use of company resources by the employees themselves. Zonk, along with other scripts referred to in this article, can be found on the author's Web site (see the inset On the Net); it also features other tools useful for network administrators.

Listing 5. fhelp.pl

#!/usr/bin/perl use Switch;
use Net::Pcap;
my $dev = "eth0"; # the interface for sniffing my $pcap;
my $err;
my $packet;
sub connread;
if ($ARGV[0] eq "-h") { print "Usage: ./fhelp.pl nr If no filename given will start live capturen";
exit;
} # check if a file name has been provided, if so open the file; # otherwise, open "live capture" if ($ARGV[0] == "") { if($pcap = Net::Pcap::open_live($dev, 2000, 1, 1000, $err)) { Net::Pcap::loop($pcap, -1, &connread, '');
} else { print "Errorn$errn";
exit;
}} else { if($pcap = Net::Pcap::open_offline($ARGV[0], $err)) { Net::Pcap::loop($pcap, -1, &connread, '');
} else { print "Errorn$errn"